Detecting Human-Object Interactions with Object-Guided Cross-Modal Calibrated Semantics

Human-Object Interaction (HOI) detection is an essential task to understand human-centric images from a fine-grained perspective. Although end-to-end HOI models thrive, their paradigm of parallel human/object and verb class prediction loses two-stage methods' merit: object-guided hierarchy. The object in one triplet gives direct clues the be predicted. In this paper, we aim boost with statistical priors. Specifically, We propose utilize Verb Semantic Model (VSM) use semantic aggregation profit Similarity KL (SKL) loss proposed optimize VSM align dataset's To overcome static embedding problem, generate cross-modality-aware visual features by Cross-Modal Calibration (CMC). above modules combined composes Object-guided Cross-modal Network (OCN). Experiments conducted on two popular benchmarks demonstrate significance incorporating prior knowledge produce state-of-the-art performances. More detailed analysis indicates serve as stronger predictor more superior method utilizing knowledge. codes are available at https://github.com/JacobYuan7/OCN-HOI-Benchmark.